Projection video display apparatus and video display method

ABSTRACT

An operation target device projects a display video onto a projection surface, which is captured by a camera. An operation detectable region specifying unit specifies respective regions where an operation is detectable and is undetectable in a range in which the display video is projected, based on image data captured by the camera. Finger and pen contact position detection units detect the operation to the operation object based on the image data. The operation target device displays the display video so that the region where the operation to the operation object is detectable, which is specified by the operation detectable region specifying unit, is distinguished from the region where the operation to the operation object is undetectable. The operation target device changes a display position of the operation object so the operation object is displayed within a range of the region where the operation to the operation object is detectable.

CROSS REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.15/524,704, filed on May 5, 2017, which claims the benefit ofPCT/JP2014/080120 filed on Nov. 13, 2014 which are incorporated byreference as if fully set forth.

TECHNICAL FIELD

The present invention relates to a projection video display apparatusand a video display method, and particularly relates to a techniqueeffective in improving a user operation with a finger, an electronicpen, or the like.

BACKGROUND ART

Many techniques have been devised with respect to an apparatus whichdisplays a video by projection. For example, an operation detectionapparatus which “detects operation information without using specialequipment” and a program thereof are described as a purpose in PatentDocument 1.

Further, as means for achieving the purpose, Patent Document 1 discloses“an operation detection apparatus used together with illumination meansand image capture means, the apparatus including: means for causing theimage capture means to capture an operator with the operator illuminatedby the illumination means; means for detecting a region of a specificpart of the operator based on image data of the operator obtained by theimage capture means; means for extracting a portion of a shadow from thedetected region of the specific part of the operator; and means fordetecting a plurality of segments each having an edge forming a straightline from the extracted portion of the shadow, detecting a point wherethe detected segments intersect one another at an acute angle, anddetecting the point of intersection as a finger pointing position withinthe region of the specific part of the operator”.

RELATED ART DOCUMENTS Patent Documents

Patent Document 1: Japanese Patent Application Laid-Open No. 2008-59283

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

In the technique of the aforementioned Patent Document 1, a fingeroperation and a finger position are detected based on a shadow of afinger which has appeared by irradiating the finger with a light or thelike. Accordingly, if the shadow of the finger has lost its shape due toan influence of surrounding light such as external light, the fingeroperation and the finger position may become undetectable.

For a user, a region where the user operation is undetectable due to aninfluence of external light or the like cannot be grasped. Therefore,there is the problem that the user himself/herself cannot improve ausage environment.

An object of the present invention is to provide a technique capable ofcorrecting false recognition and non-detection of a user operation dueto an influence of external light or the like.

The above and other objects and novel characteristics of the presentinvention will be apparent from the description of the presentspecification and the accompanying drawings.

Means for Solving the Problems

The following is a brief description of an outline of the typicalinvention disclosed in the present application.

Namely, a typical projection video display apparatus is a video displayapparatus which is controllable by an operation to an operation objectprojected onto a projection surface. The projection video displayapparatus includes: a projection unit; an image capture unit; anoperable region specifying unit; and a detection unit.

The projection unit projects a display video onto the projectionsurface. The image capture unit captures the projection surface. Theoperable region specifying unit specifies each of a first region wherethe operation to the operation object is detectable and a second regionwhere the operation to the operation object is undetectable in a rangein which the display video is projected on the projection surface, basedon image data captured by the image capture unit. The detection unitdetects the operation to the operation object based on the image datacaptured by the image capture unit.

Then, the projection unit can project a display video in which the firstregion where the operation to the operation object is detectable, whichis specified by the operable region specifying unit, is displayed so asto be distinguished from the second region. Also, the projection unitchanges a display position of the operation object so that the operationobject is displayed within a range of the first region where theoperation to the operation object is detectable.

In particular, when an input video input from outside is projected fromthe projection unit and displayed, the projection unit further changes arange occupied by the input video in the display video based on thefirst region specified by the operable region specifying unit.

In addition, the operable region specifying unit specifies each of thefirst region where the operation to the operation object is detectableand the second region where the operation to the operation object isundetectable, based on a luminance level of image data of the projectionsurface captured by the image capture unit.

Effects of the Invention

The effects obtained by the typical invention disclosed in the presentapplication will be briefly described below.

It is possible to improve the user convenience.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 is an explanatory view illustrating a configuration example of anoperation detection apparatus according to a first embodiment;

FIG. 2 is an explanatory view illustrating another configuration exampleof the operation detection apparatus illustrated in FIG. 1;

FIG. 3 is an explanatory view illustrating an example of an overview ofthe operation detection apparatus illustrated in FIG. 1 and anappearance of a user who operates the operation detection apparatus;

FIG. 4 is an explanatory view illustrating a configuration example of anelectronic pen included in the operation detection apparatus illustratedin FIG. 1;

FIG. 5 is an explanatory view illustrating an example of the state wherelight from outside is irradiated onto a projection surface at the timeof image projection, which has been examined by the inventors of thepresent invention;

FIG. 6 is a flowchart illustrating an example of a region specifying andpresentation process by the operation detection apparatus illustrated inFIG. 1;

FIG. 7 is an explanatory view illustrating a configuration example of anoperation detection determination table used in a process in step S101in the flowchart illustrated in FIG. 6;

FIG. 8 is an explanatory view illustrating an example of display of aregion where a finger operation is undetectable and a guide display to auser by the operation detection apparatus illustrated in FIG. 1;

FIG. 9 is an explanatory view illustrating another example of display ofa region where a finger operation is undetectable and a guide display toa user by the operation detection apparatus illustrated in FIG. 1;

FIG. 10 is an explanatory view illustrating an example of display of aregion where a finger operation is undetectable, a guide display to auser, and display of an operation menu icon by the operation detectionapparatus illustrated in FIG. 1;

FIG. 11 is an explanatory view illustrating an example of display of aregion where a finger operation is undetectable, a change in aprojection range, and display of an operation menu icon by the operationdetection apparatus illustrated in FIG. 1;

FIG. 12 is an explanatory view illustrating a configuration example ofan operation detection apparatus according to a third embodiment;

FIG. 13 is a flowchart illustrating an example of a process by theoperation detection apparatus illustrated in FIG. 12;

FIG. 14 is an explanatory view illustrating an example of video displayat the time of video projection onto a desk by the operation detectionapparatus illustrated in FIG. 12;

FIG. 15 is an explanatory view illustrating a configuration example ofan operation detection apparatus according to a fourth embodiment; and

FIG. 16 is an explanatory view illustrating a configuration example ofan operation detection apparatus according to a fifth embodiment.

DESCRIPTIONS OF THE PREFERRED EMBODIMENTS

In the embodiments described below, the invention will be described in aplurality of sections or embodiments when required as a matter ofconvenience. However, these sections or embodiments are not irrelevantto each other unless otherwise stated, and the one relates to the entireor a part of the other as a modification example, details, or asupplementary explanation thereof.

Also, in the embodiments described below, when referring to the numberof elements (including number of pieces, values, amount, range, and thelike), the number of the elements is not limited to a specific numberunless otherwise stated or except the case where the number isapparently limited to a specific number in principle, and the numberlarger or smaller than the specified number is also applicable.

Further, in the embodiments described below, it goes without saying thatthe components (including element steps) are not always indispensableunless otherwise stated or except the case where the components areapparently indispensable in principle.

Similarly, in the embodiments described below, when the shape of thecomponents, positional relation thereof, and the like are mentioned, thesubstantially approximate and similar shapes and the like are includedtherein unless otherwise stated or except the case where it isconceivable that they are apparently excluded in principle. The samegoes for the numerical value and the range described above. Also,components having the same function are denoted by the same referencecharacters throughout the drawings for describing the embodiments, andthe repetitive description thereof is omitted.

Hereinafter, embodiments will be described in detail.

First Embodiment

<Configuration Example of Operation Detection Apparatus>

FIG. 1 is an explanatory view illustrating a configuration example of anoperation detection apparatus 100 according to the first embodiment.

The operation detection apparatus 100 which is a projection videodisplay apparatus includes a coordinate detection device 101, anoperation target device 150, and an electronic pen 400 as illustrated inFIG. 1. The coordinate detection device 101 is configured of a camera102, illuminations 103 and 104, a finger contact position detection unit105, a pen contact position detection unit 106, an operation detectableregion specifying unit 107, a communication unit 108, and a control unit109.

The camera 102 serving as an image capture unit is configured of animage sensor, a lens, a filter and others, and captures an image. Eachof the two illuminations 103 and 104 is configured of a light emittingdiode, a circuit board, a lens and others (not illustrated).

The illuminations 103 and 104 each irradiate a region to be captured bythe camera 102. The illuminations 103 and 104 may be configured to blinkso as to be alternately turned on, other than being always turned on.

In addition to this, both the illumination 103 and the illumination 104may be temporarily turned off when lighting is switched therebetween.Alternatively, the illumination 103 and the illumination 104 may beconfigured to blink at the same timing.

Also, light from the illuminations 103 and 104 may be invisible. Forexample, the camera 102 and the illuminations 103 and 104 may berespectively configured of an infrared camera and infrared illuminationsso that the process of detecting a contact position of a finger isperformed by capturing an infrared image. If the camera 102 and theilluminations 103 and 104 are respectively configured of an infraredcamera and infrared illuminations, a filter may be added to theconfiguration of the infrared camera so as to capture an infrared imagewhile blocking a part or the whole of the light other than that in aninfrared region.

The finger contact position detection unit 105 serving as a detectionunit detects a contact position of a finger from image data captured bythe camera 102. The pen contact position detection unit 106 similarlyserving as a detection unit detects a contact position of the electronicpen 400 from the image data captured by the camera 102.

The detection process by the finder contact position detection unit 105and the pen contact position detection unit 106 is performed based onsoftware in a program format by an electronic circuit and the likeconstituting the finder contact position detection unit 105 and the pencontact position detection unit 106.

Contact position information detected by the finger contact positiondetection unit 105 or the pen contact position detection unit 106 isprocessed as operation information of the finger or the electronic pen400 by a control unit 153.

Hereinafter, in the present embodiment, “finger” is described as arepresentative for an object to be detected. An object whose contactposition is to be detected by the finger contact position detection unit105 may be a pen-shaped object such as a pointing stick in addition tothe finger. Further, it is sufficient if at least one of the fingercontact position detection unit 105 and the pen contact positiondetection unit 106 is provided in this configuration, and it is notalways necessary to provide both of them.

The operation detectable region specifying unit 107 serving as anoperable region specifying unit specifies a region where operationinformation of the finger or the electronic pen 400 is detectable, fromthe image data captured by the camera 102. The region where theoperation information is detectable serves as a first region.

The region specifying process is performed based on software in aprogram format by an electronic circuit and the like constituting theoperation detectable region specifying unit 107. Note that a techniquefor specifying the region will be described below with reference toFIGS. 6 and 7.

The communication unit 108 is an interface configured of a networkconnection, a USB (Universal Serial Bus) connection, an ultrasonic unit,an infrared communication device and others, and communicating with acommunication unit 151 included in the operation target device 150.

This communication unit 108 performs communication of detection resultdata 120 or the like. The detection result data 120 is information whichis output from the coordinate detection device 101 to the operationtarget device 150 via the communication unit 108. Specific examplesthereof include contact point coordinates of the finger, contact pointcoordinates of the electronic pen 400, and finger/pen operationinformation undetectable region data.

The finger/pen operation information undetectable region data includesinformation representing a region where operation information of afinger/electronic pen is undetectable, that is, positional information(for example, coordinate data) representing a region where an operationwith a finger/electronic pen is unusable and information representing adegree related to a detection accuracy in a case where operationinformation of a finger/pen is undetectable. A region where an operationis unusable serves as a second region.

The control unit 109 controls the camera 102, the illumination 103, theillumination 104, the finger contact position detection unit 105, thepen contact position detection unit 106, the operation detectable regionspecifying unit 107, and the communication unit 108. The control unit109 is configured of, for example, a CPU (Central Process Unit), and thecontrol unit 109 performs control based on the software in a programformat.

Next, the operation target device 150 serving as a projection unit isconfigured of the communication unit 151, a setting input unit 152, thecontrol unit 153, a video irradiation unit 154, a display unit 155, alight emitting unit 156, and an audio output unit 157.

The operation target device 150 is, for example, a projector thatdisplays drawing information of the finger of a user and performsoperation, setting, and control of the projector itself by receiving thedetection result data 120 transmitted from the communication unit 108 ofthe coordinate detection device 101.

The communication unit 151 is an interface configured of a networkconnection, a USB (Universal Serial Bus) connection, an ultrasonic unit,an infrared communication device and others like the communication unit108, and communicating with the coordinate detection device 101. Thesetting input unit 152 is an interface configured of a button, a touchpanel and others and receiving a user input.

The control unit 153 is configured of, for example, a CPU and controlsthe communication unit 151, the setting input unit 152, the videoirradiation unit 154, the display unit 155, the light emitting unit 156,the audio output unit 157, the electronic pen 400, and the like. Thecontrol unit 153 performs control based on the software in a programformat.

The video irradiation unit 154 is configured of a light source lamp, aliquid crystal panel, a lens and others, and irradiates a video onto aprojection surface 301 illustrated in FIG. 3. The video irradiated ontothe projection surface 301 is output from a personal computer (notillustrated) or the like connected to the operation target device 150.

Here, if light in a visible region having a wavelength shorter than apredetermined wavelength is set as light irradiated from the videoirradiation unit 154 and light in an invisible region having awavelength longer than the predetermined wavelength is set as lightirradiated from the illuminations 103 and 104, the user can visuallyrecognize only the light irradiated from the video irradiation unit 154.A band-pass filter or the like may be used for controlling thewavelength of the irradiated light.

The display unit 155 is configured of a liquid crystal display, anelectronic circuit which drives the liquid crystal display and others,and displays information such as characters and images. The lightemitting unit 156 is configured of a light emitting element, anelectronic circuit which drives the light emitting element and others,and creates states such as lighting, blinking, and non-lighting.

The audio output unit 157 is configured of a speaker, an amplificationcircuit which amplifies a signal or the like to be output to the speakerand others, and outputs various audios such as a voice message and asound effect. The electronic pen 400 is configured of a light emittingelement, an electronic circuit which drives the light emitting elementand others.

As a specific process of the control unit 153, the control unit 153performs video projection control to the video irradiation unit 154 soas to be able to distinguish the region where an operation with a fingeris detectable and the region where an operation with a finger isundetectable. Alternatively, the video projection control may beperformed so as to be able to distinguish the regions in n levels inaccordance with the degree of detection accuracy instead of the twolevels of the region where an operation with a finger is detectable andthe region where an operation with a finger is undetectable. Here, n is2, 3, . . . n.

Furthermore, the control unit 153 performs the video projection controlto the video irradiation unit 154 so that a menu icon, a user messageand the like are displayed on the projection surface 301 while avoidingthe region where an operation with a finger is undetectable. Inaddition, the control unit 153 performs control to adjust the size of avideo projected onto the projection surface 301 to the video irradiationunit 154 so as to eliminate the region where an operation with a fingeris undetectable.

Note that functional block units such as the camera 102, theilluminations 103 and 104, the finger contact position detection unit105, the pen contact position detection unit 106, the operationdetectable region specifying unit 107, the communication unit 108, andthe control unit 109 are independent of one another in FIG. 1, but maybe constituted of one or plural components, as needed.

For example, the finger contact position detection unit 105, the pencontact position detection unit 106, and the operation detectable regionspecifying unit 107 may be configured to perform the processes thereofby one or plural central process units (CPUs) or the like.

Also, in FIG. 1, the coordinate detection device 101 is configured toinclude all functional block units such as the camera 102, theilluminations 103 and 104, the finger contact position detection unit105, the pen contact position detection unit 106, the operationdetectable region specifying unit 107, the communication unit 108, andthe control unit 109.

Similarly, the operation target device 150 is configured to include allfunctional block units such as the communication unit 151, the settinginput unit 152, the control unit 153, the video irradiation unit 154,the display unit 155, the light emitting unit 156, and the audio outputunit 157.

However, the aforementioned configuration may be replaced with aconfiguration in which one or plural components are formed outside andare connected via a network connection, a USB connection, or the like asillustrated in FIG. 2 to be described below.

<Another Configuration Example of Operation Detection Apparatus>

FIG. 2 is an explanatory view illustrating another configuration exampleof the operation detection apparatus 100 illustrated in FIG. 1.

In this case, the operation detection apparatus 100 has a configurationin which each of the camera 102 and the illuminations 103 and 104 isindividually provided independently of the coordinate detection device101 as illustrated in FIG. 2.

As still another configuration example of the operation detectionapparatus 100, the coordinate detection device 101 and the operationtarget device 150 may be configured as one device.

<Overview and Operation Example of Operation Detection Apparatus>

FIG. 3 is an explanatory view illustrating an example of an overview ofthe operation detection apparatus 100 illustrated in FIG. 1 and anappearance of a user who operates the operation detection apparatus 100.FIG. 3(a) illustrates an example in a case where a user 300 performs anoperation with his/her finger, and FIG. 3(b) illustrates an example in acase where the user 300 performs an operation with the electronic pen400.

In FIG. 3, the camera 102 captures an image of a range indicated by aprojection range 302 on a projection surface 301. Any of a wall, ascreen, or a desk may be used as the projection surface 301.

When the user 300 touches the projection range 302 on the projectionsurface 301 with his/her finger as illustrated in FIG. 3(a), the fingercontact position detection unit 105 in the operation detection apparatus100 detects a contact position of the finger. Also, when the user 300touches the projection range 302 on the projection surface 301 with theelectronic pen 400 as illustrated in FIG. 3(b), the pen contact positiondetection unit 106 detects a contact position of the pen.

Further, the finger contact position detection unit 105 may have afunction of detecting a shape and movement of the finger of the userfrom image data captured by the camera 102 when the user 300 hasperformed a gesture operation in the vicinity of the projection surface301.

Information detected by the finger contact position detection unit 105is transferred as the detection result data 120 to the operation targetdevice 150 via the communication unit 108. Then, the control unit 153controls the operation target device 150 in accordance with thedetection result data 120.

In this manner, the contact position of the electronic pen or the fingerwith an operation surface can be correctly detected without a touchsensor or the like provided on the operation surface.

<Configuration Example of Electronic Pen>

FIG. 4 is an explanatory view illustrating a configuration example ofthe electronic pen 400 included in the operation detection apparatus 100illustrated in FIG. 1. FIG. 4(a) illustrates an example of an appearanceof the electronic pen 400. FIG. 4(b) illustrates an example in a casewhere a tip of the electronic pen 400 is not in contact, and FIG. 4(c)illustrates an example in a case where the tip of the electronic pen 400is in contact.

The electronic pen 400 has a light emitting element 401 at its tipportion, and a tip contact detection unit 402 and a light emissioncontrol unit 403 are provided in a grip portion of the electronic pen400 as illustrated in FIG. 4(a). The tip contact detection unit 402 isconfigured of, for example, a pressure sensitive sensor and anelectronic circuit and determines whether the tip of the electronic pen400 contacts a wall surface or the like.

The light emission control unit 403 performs control so that the lightemitting element 401 is turned off when the tip of the electronic pen400 is not in contact as illustrated in FIG. 4(b), based on adetermination result of the tip contact detection unit 402.

Also, the light emission control unit 403 performs control so that thelight emitting element 401 is turned on when the tip contact detectionunit 402 determines that the tip of the electronic pen 400 is in contactas illustrated in FIG. 4(c).

Alternatively, control may be performed so that the light emittingelement 401 enters different states between the time of contact and thetime of non-contact. For example, control may be performed so that thelight emitting element 401 is turned on when the tip of the electronicpen 400 is in contact and blinks when the tip of the electronic pen 400is not in contact. Further, control may be performed so that the lightemitting element 401 blinks at different frequencies between the time ofcontact and the time of non-contact.

Further, the camera 102 and the light emitting element 401 may berespectively configured of an infrared camera and an infraredillumination so that the process of detecting a contact position of theelectronic pen 400 is performed by capturing an infrared image. Further,if the camera 102 and the light emitting element 401 are respectivelyconfigured of an infrared camera and an infrared illumination, a filtermay be added to the configuration of the infrared camera so as tocapture an infrared image while blocking apart or the whole of the lightother than that in an infrared region.

Subsequently, a technique in which the operation detection apparatus 100illustrated in FIG. 1 specifies a region where operation information ofa finger is undetectable, presents the region to the user, and displaysa guide for encouraging an improvement will be described.

<Influence of Irradiation of External Light>

FIG. 5 is an explanatory view illustrating an example of the state wherelight from outside is irradiated onto a projection surface at the timeof image projection, which has been examined by the inventors of thepresent invention.

External light 701 illustrated in FIG. 5 is light from outside such aslight from a fluorescent lamp or an incandescent lamp or solar light,and is a light source different from the illuminations 103 and 104. Ifthe external light 701 is heavily irradiated onto the projection surface301, shadows 703 and 704 respectively formed by the illuminations 103and 104 have lost their shapes or fade and a light-dark contrast fromsurroundings becomes less distinguishable.

Consequently, phenomena of failing to accurately catch the shapes by thecamera 102 and failing to accurately detect the contact position of afinger 702 on the projection surface 301 by the finger contact positiondetection unit 105 occur.

Thus, the operation detection apparatus 100 illustrated in FIG. 1performs a region specifying and presentation process. The regionspecifying and presentation process specifies a region where the contactposition of the finger 702 on the projection surface 301 cannot beaccurately detected due to the external light 701 or the like asillustrated in FIG. 5, that is, a region where operation information ofa finger is undetectable. Then, the specified region is presented to theuser, and a guide for encouraging an improvement is displayed.

In this manner, when operation information of a finger is undetectable,an undetectable location on the projection surface can be clearly shownto the user, so that the operability can be improved.

<Process Example of Region Specifying and Presentation Process>

FIG. 6 is a flowchart illustrating an example of the region specifyingand presentation process by the operation detection apparatus 100illustrated in FIG. 1.

FIG. 6 illustrates a flow including a series of processes in steps S101to S106 each executed once, but this flow is continuously repeated inpractice. Accordingly, after the process in step S103 or S106, the flowreturns to the process in step S101 again.

The flowchart illustrated in FIG. 6 is always executed while the powerto the operation detection apparatus 100 is turned on, for example,before the user starts to use the operation detection apparatus 100 andduring the use of the operation detection apparatus 100.

Accordingly, even when an ambient environment of the operation detectionapparatus 100 has suddenly changed, for example, when luminance ofexternal light has suddenly changed while the user is using theoperation detection apparatus 100, an effect can be obtained.

Also, the flowchart illustrated in FIG. 6 may be executed only when afinger operation function by the user is used. Namely, when the fingeroperation function by the user has been invalidated and the operationtarget device 150 is used only for the purpose of projecting a video,the process illustrated in FIG. 6 is not performed. In this manner, itis possible to achieve an effect of reducing the process load and powerconsumption of a CPU, that is, the control unit 153 or the like in theoperation detection apparatus 100.

First, the operation detectable region specifying unit 107 performs theprocess of determining whether an operation with a finger is detectablebased on image data of the projection surface 301 captured by the camera102 (step S101).

An example of the determination process includes a determinationtechnique using a luminance level. In a situation where only theilluminations 103 and 104 are turned on and the external light 701 isnot irradiated at all, luminance of image data captured by the camera102 is previously retained as data, and if luminance of image data beingcurrently captured is higher than that of the retained data by apredetermined value (first setting value) or more, it is possible topresume that the amount of the external light 701 is large. As a result,operation information of a finger is undetectable, and it is thuspossible to determine that the operation with a finger is unusable.

Alternatively, if image data is captured in the state where theilluminations 103 and 104 are temporarily turned off, that is, only theexternal light 701 is irradiated, and luminance of the image data atthat time is higher than a predetermined value (first setting value), itis possible to determine that the amount of the external light 701 islarge and the operation with a finger is unusable. This determinationprocess will be described with reference to FIG. 7.

Subsequently, the control unit 153 determines whether a finger operationis usable based on a result of the process in step S101, that is,finger/pen operation information undetectable region data in thedetection result data 102 (step S102). If it is determined that thefinger operation is usable (Yes) in the process in step S102, the flowproceeds to the process in step S103.

If it is determined that the finger operation is usable in the processin step S102, the control unit 153 displays a notification that the usercan use the operation with a finger via the video irradiation unit 154in the projection range 302 (step S103), and the flow ends.

Note that the process in step S103 may not be performed initially.Namely, this may be notified to the user only when a finger operationbecomes unusable once while the apparatus is in use and the fingeroperation then becomes usable.

On the other hand, if it is determined that the finger operation isunusable (No) in the process in step S102, the flow proceeds to theprocess in step S104. In this process, the control unit 153 displays aregion where a finger operation is undetectable via the videoirradiation unit 154 on the projection surface 301 (step S104). Thedisplay of the region where a finger operation is undetectable will bedescribed below with reference to FIG. 8.

Subsequently, the control unit 153 determines a display position of anoperation object based on the region where a finger operation isundetectable (step S105). The determination of the display position willbe described below with reference to FIGS. 9 and 10. The operationobject mentioned here represents a display of a message to the user andan operation menu icon to be projected onto the projection surface 301.

Then, the control unit 153 performs guide display for encouraging theuser to re-examine an ambient environment in which the operationdetection apparatus 100 has been set, via the video irradiation unit 154to the projection surface 301 (step S106). Note that the guide displaywill be described below with reference to FIGS. 8 and 9.

In the above-described manner, the region specifying and presentationprocess by the operation detection apparatus 100 ends. After the processin step S106 ends, the flow returns to the process in step S101 again,and the flow illustrated in FIG. 6 is repeated.

If the user selects continuation of the process without re-examining theambient environment after the process in step S106, the flow endswithout returning to step S101.

<Example of Process in Step S101 and Configuration Example of OperationDetection Determination Table>

Subsequently, determination process in step S101 illustrated in FIG. 6will be additionally described.

FIG. 7 is an explanatory view illustrating a configuration example of anoperation detection determination table 900 used in the process in stepS101 in the flowchart illustrated in FIG. 6.

The operation detectable region specifying unit 107 determines whetheran operation with a finger is detectable by the use of the operationdetection determination table 900 illustrated in FIG. 7. The operationdetection determination table 900 is stored in a storage unit (notillustrated) or the like included in the operation detectable regionspecifying unit 107.

The operation detection determination table 900 includes an index number901, a finger operation detection accuracy 902, a luminance level 903, areference luminance difference 904, and a display color 905 asillustrated in FIG. 7.

The index number 901 represents an index number of the operationdetection determination table 900. The finger operation detectionaccuracy 902 is an index representing a detection accuracy of anoperation with a finger. Since the detection accuracy of the operationwith a finger becomes higher as the numerical value of the index becomeslarger, the value of 3 indicates that the finger operation is mostdetectable and the value of 0 indicates that the finger operation isleast detectable.

In the example of the operation detection determination table 900illustrated in FIG. 7, the value of 3 indicates that the operation witha finger can be detected, and the other values of 2 to 0 indicate thatthe operation with a finger cannot be detected.

The luminance level 903 is associated with the finger operationdetection accuracy 902, and represents a range of a luminance levelvalue of image data captured by the camera 102. The luminance levelvalue is expressed by 256 levels from 0 to 255 as an example, andindicates that the image data is the brightest when the luminance levelvalue is 255 and that the image data becomes darker as the luminancelevel value comes closer to 0.

Accordingly, it is determined that the image data is more affected byexternal light as the luminance level value comes closer to 255, and thefinger operation detection accuracy 902 comes close to 0. On the otherhand, it is determined that the image data is less affected by externallight as the luminance level value comes closer to 0, and the fingeroperation detection accuracy 902 comes close to 3.

The reference luminance difference 904 is associated with the fingeroperation detection accuracy 902, and represents a range of a valuetaken as a difference between the luminance level value of the imagedata captured by the camera 102 and a reference luminance level valueretained in the apparatus. The reference luminance level value retainedin the apparatus is a value in a case where the image data is notaffected by external light, and thus can be used to guess how the imagedata is affected by external light by comparison with a luminance levelvalue of image data captured in a usage environment.

Accordingly, it is determined that the image data is more affected byexternal light as a value of the reference luminance difference 904becomes larger, and the finger operation detection accuracy 902 comesclose to 0. Meanwhile, it is determined that the image data is lessaffected by external light as a value of the reference luminancedifference 904 becomes smaller, and the finger operation detectionaccuracy 902 comes close to 3.

Note that the finger operation detection accuracy 902 may be determinedbased on a condition of either one of the luminance level 903 and thereference luminance difference 904, or may be determined based on anyoneof an AND (logical product) condition, an OR (logical add) condition,and an XOR (exclusive logical add) condition of both the luminance level903 and the reference luminance difference 904.

The display color 905 represents a color displayed on the projectionsurface 301 in association with the finger operation detection accuracy902. The color represented by the display color 905 is displayed on theprojection surface 301 for the region where an operation with a fingeris undetectable determined in the process in step S105 illustrated inFIG. 6, in other words, in accordance with the value of the fingeroperation detection accuracy 902.

Note that colors respectively represented in the display color 905 areexpressed by screening, dots and hatching in FIG. 7. When the value ofthe finger operation detection accuracy 902 is 3, the display color 905is set to be colorless, for example. Alternatively, the display color905 may be expressed by a single color instead of different colors. Inthis case, the display color 905 is expressed by color density.

Note that FIG. 7 illustrates an example of a table having four entries,but a table having a different number of entries may be used. If thetable is set so as to satisfy the condition of the number of entries 2,respective areas where an operation with a finger is detectable and isundetectable can be specified.

Accordingly, the number of entries, that is, the number of indexes ofthe operation detection determination table 900 is not limited as longas the number of entries 2 is satisfied. Note that this determinationprocess may be applied to all pixels within the projection range 302 onthe projection surface 301, or may be applied to only some of thepixels. Further, this determination process may be applied aftercalculation of an average value or the like is performed for a set ofcollected m×n pixels. Here, m and n are respectively any integer values.

<Example of Display of Region where Finger Operation is Undetectable andGuide Display to User>

Subsequently, display on the projection surface 301 related to theprocesses in step S103 to step S106 in the flowchart illustrated in FIG.6 will be described below with reference to FIGS. 8 to 10.

FIG. 8 is an explanatory view illustrating an example of display of aregion where a finger operation is undetectable and a guide display to auser by the operation detection apparatus 100 illustrated in FIG. 1.

FIG. 8(a) illustrates a display example on the projection surface 301for guiding re-examination of an ambient environment to the user afterthe region where an operation with a finger is undetectable is specifiedin the process in step S104 and the process in step S106 illustrated inFIG. 6.

In FIG. 8, a region 1000 represents a region where an operation with afinger is undetectable, and is displayed so that the user understandsdetection accuracy by using the color expression defined in the displaycolor 905 illustrated in FIG. 7. A region 1001 represents a region wherean operation with a finger is detectable, and is displayed in a colordefined in the display color 905 illustrated in FIG. 7. A guide display1002 represents display for encouraging the user to improve a usageenvironment.

When regions where an operation with a finger is detectable and isundetectable are respectively colored in different colors and presentedto the user in the region 1000 and the region 1001, different colors maybe applied to each pixel or different colors may be applied to each setof collected a×b pixels. Here, a and b are respectively any integervalues.

Furthermore, different colors may be applied to each of a predeterminednumber of regions obtained by dividing the projection range 302. Whendifferent colors are applied to each set of collected a×b pixels or toeach of the plurality of regions obtained by dividing the projectionrange 302, the amount of information notified to the user can bereduced. Thus, the user easily recognizes a current state, and as aresult, user convenience can be enhanced.

FIG. 8(b) illustrates a display example in a case where a user hasre-examined a usage environment by, for example, blocking solar lightwith a curtain and a certain degree of improvement has been made fromthe state illustrated in FIG. 8(a).

It is observed that a part of the region 1000 where an operation with afinger is undetectable illustrated in FIG. 8(a) is changed into theregion 1001 where an operation is detectable, and a detection accuracyof the region 1000 where an operation is undetectable is improved.

FIG. 8(c) illustrates a display example in a case where a user hasfurther re-examined a usage environment and a further improvement hasbeen made from the state illustrated in FIG. 8(b), so that the region1000 where an operation with a finger is undetectable has beencompletely eliminated from the projection range 302.

The region 1000 where an operation with a finger is undetectable isentirely changed into the region 1001 where an operation is detectable,and the guide display 1002 is changed to a sentence indicating that anoperation with a finger can be detected in an entire region of theprojection range 302.

<Another Example of Guide Display>

FIG. 9 is an explanatory view illustrating another example of thedisplay of a region where a finger operation is undetectable and theguide display to the user by the operation detection apparatus 100illustrated in FIG. 1.

FIG. 9 illustrates an example in which display positions of the guidedisplay 1002 to the user and an operation button display 1100 are madevariable depending on the region where an operation with a finger isundetectable in the process in step S105 and the process in step S106illustrated in FIG. 6.

In this case, the guide display 1002 and the operation button display1100 are displayed within the region 1001 where an operation with afinger is detectable while avoiding the region 1000 where an operationwith a finger is undetectable as illustrated in FIGS. 9(a) and 9(b).

This can solve the problem of inability to select an operation button bya finger operation and the problem of making it difficult to see thedisplay due to the overlap between the display of a region where anoperation with a finger is undetectable and the user message display. Asa result, user convenience can be improved.

In this case, the guide display 1002 may be displayed on any coordinateson the projection surface 301, and only the operation button display1100 by which the user operates the projection surface 301 with his/herfinger or gesture may be displayed within the region 1001 where anoperation with a finger is detectable.

<Display Example of Operation Menu Icon or the Like>

FIG. 10 is an explanatory view illustrating an example of display of aregion where a finger operation is undetectable, a guide display to theuser, and display of an operation menu icon by the operation detectionapparatus 100 illustrated in FIG. 1.

FIG. 10 illustrates a display example in which a display position of anoperation menu icon 1200 is made variable depending on the region wherean operation with a finger is undetectable in the process in step S105and the process in step S106 illustrated in FIG. 6.

As illustrated in each of FIGS. 10(a) and 10(b), the operation menu icon1200 is displayed within the region 1001 where an operation with afinger is detectable while avoiding the region 1000 where an operationwith a finger is undetectable.

This can solve the problem of inability to select the operation menuicon 1200 by a finger operation and the problem of making it difficultto see the display of the menu icon due to the overlap between thedisplay of a region where an operation with a finger is undetectable andthe menu icon. As a result, user convenience can be improved.

Note that, if the region where an operation with a finger isundetectable is wide and the region where the guide display 1002 to theuser is displayed is insufficient, the guide display may be displayed byreducing the font size thereof only in that case.

Alternatively, a notification may be made to the user by an icon byprojecting the icon from the video irradiation unit 154. Also, thenotification may be made to the user by displaying a guide on thedisplay unit 155. Further, the notification may be made to the user byoutputting a voice guide and a sound effect from the audio output unit157.

Furthermore, if the region where an operation with a finger isundetectable is wide and the region where the operation menu icon 1200is displayed is insufficient, the operation menu icon 1200 may also besimilarly displayed by reducing the display size thereof. Alternatively,the icons may be displayed by reducing the number of types thereof to aminimum number.

This can solve the problem of inability to display the guide display1002 to the user due to the region where an operation with a finger isundetectable being too wide. As a result, user convenience can beimproved.

As described above, the user can re-examine the usage environment of theoperation target device while recognizing the region where operationinformation of a finger is undetectable and confirming an improvementstatus of the region. Thus, user convenience for the operation detectionapparatus 100 can be improved.

Second Embodiment

<Outline>

In the second embodiment, a process of adjusting a projection range ofvideo projection to exclude the region where operation information of afinger is undetectable from the projection surface when the region whereoperation information of a finger is undetectable is present or when itis difficult to eliminate the region where operation information of afinger is undetectable from the projection surface in the operationdetection apparatus 100 will be described. This process is performed bythe control unit 153 based on finger/pen operation informationundetectable region data in the detection result data 120.

<Display Example of Operation Menu Icon or the Like>

FIG. 11 is an explanatory view illustrating an example of display of aregion where a finger operation is undetectable, a change in aprojection range, and display of an operation menu icon by the operationdetection apparatus 100 illustrated in FIG. 1.

FIG. 11 illustrates an example of a projection surface in a case wherethe region where operation information of a finger is undetectable hasbeen eliminated from the projection surface by changing the projectionrange.

For example, as illustrated in FIG. 11(a), a region where operationinformation of a finger is undetectable like the region 1000 is presentwithin the projection range 302 on the projection surface in some cases.In such a case, the region where operation information of a finger isundetectable can be excluded from the projection range 302 by changingthe projection range 302 so as to avoid the region 1000 as illustratedin FIG. 11(b).

This process may be automatically performed when the presence of theregion where operation information of a finger is undetectable isdetected, or may be performed only when a user has permitted the changein the projection range by displaying a user message 1301 and a userselection button 1302 illustrated in FIG. 11(a).

Also, display coordinates may be changed in accordance with a state ofthe projection surface so that the user message 1301, the user selectionbutton 1302, and the operation menu icon 1200 illustrated in FIG. 11(a)are not displayed in the region 1000 where operation information of afinger is undetectable.

Furthermore, this process may be performed only when the fingeroperation function by the user has been validated. Meanwhile, when thefinger operation function has been invalidated, even if the region whereoperation information of a finger is undetectable is present, this isirrelevant to user convenience. Thus, there is no problem even if avideo is projected in any projection range determined by the user.

Also, in FIG. 11(a), the user message 1301 may be displayed on anycoordinates on the projection surface, and the display coordinates maybe changed in accordance with the state of the projection surface sothat only the user selection button 1302 and the operation menu icon1200 which the user operates with his/her finger or gesture are notdisplayed in the region 1000 where operation information of a finger isundetectable.

Furthermore, a projection size after the change may be presented to theuser in advance by projecting an outer frame of the projection size sothat the user knows the projection size after the change. Accordingly,the user easily determines whether the projection size is to be changed,so that convenience can be improved. In that case, the region 1000 whereoperation information of a finger is undetectable may not be displayed.

As described above, since it is possible to exclude the region whereoperation information of a finger is undetectable from the projectionrange by adjusting the projection range of the video when the regionwhere operation information of a finger is undetectable is present andit is difficult to eliminate the region where operation information of afinger is undetectable from the projection range, user convenience canbe improved.

Third Embodiment

<Outline>

In the third embodiment, a technique in which, when a region whereoperation information of a finger is undetectable is present or when aset projection range does not entirely fall within a projection objectsuch as a desk or a wall in the operation detection apparatus 100, theregion is detected by the same process based on luminance leveldetermination and a process corresponding to each region is performedwill be described.

<Configuration Example of Operation Detection Apparatus>

FIG. 12 is an explanatory view illustrating a configuration example ofan operation detection apparatus 100 according to the third embodiment.

The operation detection apparatus 100 illustrated in FIG. 12 differsfrom the operation detection apparatus 100 illustrated in FIG. 1according to the first embodiment described above in that a videoirradiation-enabled region specifying unit 1601 is newly added and videoirradiation-disabled region data is added as the detection result data120. The configuration other than those is similar to that illustratedin FIG. 1, and hence repetitive description thereof is omitted.

The video irradiation-enabled region specifying unit 1601 specifies aregion where a video can be irradiated onto a projection object such asa desk or a wall based on image data of the projection surface 301captured by the camera 102. The specifying technique will be describedbelow with reference to a flowchart illustrated in FIG. 13.

Also, the video irradiation-disabled region data added to the detectionresult data 120 is data representing the region where a video cannot beirradiated, which has been specified by the video irradiation-enabledregion specifying unit 1601, and includes positional information (forexample, coordinate data) representing the region where a video cannotbe irradiated, for example.

<Example of Process for Specifying Irradiation-Enabled Region>

FIG. 13 is a flowchart illustrating an example of the process by theoperation detection apparatus 100 illustrated in FIG. 12. FIG. 13illustrates a flow including a series of processes in steps S201 to S204and steps S104 to S106 each executed once, but this flow is continuouslyrepeated in practice, and the flow returns to the process in step S201after the processes in step S205, step S103 and step S106.

The flowchart illustrated in FIG. 13 is always executed while the powerto the operation detection apparatus 100 is turned on, for example,before the user starts to use the operation detection apparatus 100 andduring the use of the operation detection apparatus 100. Thus, even if asurrounding usage environment has suddenly changed, for example, achange in luminance of external light has occurred when the user isusing the apparatus, operability can be immediately improved.

Also, the flowchart illustrated in FIG. 13 may be executed only when afinger operation function by the user is used. In this manner, it ispossible to achieve an effect of reducing the process load and powerconsumption of the control unit 153 (CPU) and others.

The processes in steps S103 to S106 illustrated in FIG. 13 are similarto the processes in steps S103 to S106 illustrated in FIG. 6, and hencerepetitive description thereof is omitted.

First, the operation detectable region specifying unit 107 performsdetermination process based on image data of the projection surface 301captured by the camera 102 (step S201). In the determination process instep S201, it is determined whether an operation with a finger can bedetected and whether a set projection range falls within a projectionobject such as a desk or a wall.

As an example of the determination process, a method using a luminancelevel described in the process in step S101 illustrated in FIG. 6 isconceivable. Also, the process for determining whether an operation witha finger can be detected is similar to that described in the process instep S101 illustrated in FIG. 6.

As the process for determining whether a projection object such as adesk or a wall is wide relative to the set projection range, forexample, a method using the same luminance level can be used. In asituation where only illuminations 103 and 104 are turned on andexternal light 701 is not irradiated at all, luminance of image datacaptured by the camera 102 is previously retained as data, and ifluminance of image data being currently captured is lower than that ofthe retained data by a predetermined value (second setting value) ormore, the region can be presumed to be a region where a projectionobject does not exist, that is, an unusable region. As a result, it ispossible to determine that a video cannot be projected onto that region.

Alternatively, if image data is captured in the state where theilluminations 103 and 104 are temporarily turned off, that is, only theexternal light 701 is irradiated, and luminance of the image data atthat time is lower than a predetermined value (second setting value),the region can be presumed to be an unusable region where a projectionobject does not exist and a video cannot be projected.

If specific numerical values are cited like those illustrated in FIG. 7,when the luminance level 903 is 0 to 49, it is possible to determinethat a projection object does not exist and a video cannot be projected.Also, when the reference luminance difference 904 is −20 (minus 20) orsmaller, it is similarly possible to determine that the projectionobject does not exist and a video cannot be projected. Further, it isalso possible to determine whether the projection object does not existand a video cannot be projected from the respective indexes of both theluminance level 903 and the reference luminance difference 904.

Subsequently, the control unit 153 determines whether an unusable regionis present (step S202). In the process in step S202, the control unit153 determines whether a video can be projected in a set projectionrange based on the process result by the process in step S201. Also, asthe process result in step S202, for example, the finger/pen operationinformation undetectable region data and the video irradiation-disabledregion data in the detection result data 120 are used.

Then, when it is determined that the video cannot be projected in theset projection range (Yes), the control unit 153 determines whether theuser has permitted the change into a projection range newly determinedso that the projection object exists for an entire region of theprojection range 302 via the setting input unit 152 (step S203).

When the user has permitted the change of the projection range (Yes),the control unit 153 newly determines a projection range so that theprojection object exists for the entire region of the projection range302, changes the projection range, and irradiates the video via thevideo irradiation unit 154 (step S205).

Meanwhile, when the user does not permit the change of the projectionrange (No) in the process in step S203 or when it is determined that thevideo can be projected in the projection range set in the process instep S202 (No), the control unit 153 determines whether a region wherean operation with a finger is undetectable due to surrounding light suchas external light is present (step S204).

When the control unit 153 determines that the region where an operationwith a finger is undetectable due to surrounding light such as externallight is present (Yes) in the process in step S204, the flow proceeds tothe process in step S104.

Meanwhile, when the control unit 153 determines that the region where anoperation with a finger is undetectable due to surrounding light such asexternal light is not present (No) in the process in step S204, the flowproceeds to the process in step S103. For the following processes insteps S103 to S106, processes similar to those described in FIG. 6 areperformed.

<Example of Video Display onto Desk>

FIG. 14 is an explanatory view illustrating an example of video displayat the time of video projection onto a desk by the operation detectionapparatus 100 illustrated in FIG. 12.

FIG. 14(a) corresponds to the processes in steps S202 and S205illustrated in FIG. 13, and illustrates the state where a part of theregion of the projection range 302 is not present on the projectionsurface 301 when a video is projected onto the desk from the operationdetection apparatus 100 and thus a region 1501 cannot be correctlyprojected onto the desk.

Then, a process for determining a new projection range is performed whenthe user operates the user selection button 1302 for selecting whetherthe projection size is changed. FIG. 14(b) illustrates a state where theuser selects the user selection button 1302 of “Yes” and the projectionrange has been adjusted to the projection range 302.

Then, when the process flow illustrated in FIG. 13 is executed againfrom the beginning and the process in step S204 illustrated in FIG. 13is performed, the region 1000 where an operation with a finger isundetectable is presented and the guide display 1002 for encouraging theuser to re-examine a usage environment is displayed as illustrated inFIG. 14(c). Also, FIG. 14(d) illustrates an example in which the region1000 where an operation with a finger is undetectable is displayed onthe projection range 302 at the time illustrated in FIG. 14(a), that is,before the projection range 302 is adjusted.

As can be seen from the comparison between FIG. 14(c) and FIG. 14(d),the region 1000 where an operation with a finger is undetectable hasbeen changed by adjusting the projection range 302.

Accordingly, there is a possibility that the region 1000 where anoperation with a finger is undetectable becomes smaller when the videoprojection range is made small. On the other hand, there is apossibility that the region 1000 where an operation with a finger isundetectable becomes larger when the video projection range is madelarge.

Therefore, if both the region where an operation with a finger isundetectable and the region where a projection object such as a deskdoes not exist are specified, the projection range is first adjusted,and the region where an operation with a finger is undetectable is thendisplayed to present a guide for improving a usage environment to theuser. In this manner, it is possible to prevent unnecessary informationfrom being presented to the user, and user convenience can be enhanced.

The case where the region where a projection object such as a desk doesnot exist is detected has been described in the third embodiment.Alternatively, if an obstacle which interrupts the video projection isdetected, the projection range may be adjusted so as to exclude theobstacle from the projection range.

As a method for detecting the obstacle, for example, a method in which athreshold value of the luminance level 903 and a threshold value of thereference luminance difference 904 for obstacle detection are previouslyset and the presence of the obstacle is determined by the comparisonwith the set threshold values is conceivable. In this manner, userconvenience can be improved like the case where a projection object doesnot exist.

In the above-described manner, the region where operation information ofa finger is undetectable and the region where an image cannot beprojected because a projection object does not exist can besimultaneously specified by using the same determination process.Accordingly, it is possible to reduce the time period required toperform the process, and it is possible for the user to shorten the timeperiod from when the power to the apparatus is turned on to when theapparatus becomes usable. Further, since a time period during which theuse of the apparatus is stopped can be shortened even if the apparatusis in use, user convenience can be improved.

Fourth Embodiment

<Configuration Example of Operation Detection Apparatus>

FIG. 15 is an explanatory view illustrating a configuration example ofan operation detection apparatus 100 according to the fourth embodiment.

The operation detection apparatus 100 illustrated in FIG. 15 differsfrom the operation detection apparatus 100 illustrated in FIG. 1 in thatthe operation detectable region specifying unit 107 is provided in theoperation target device 150 instead of the coordinate detection device101 and a camera acquired image sending unit 110 is newly added to thecoordinate detection device 101. In addition, the detection result data121 to be transmitted and received between the coordinate detectiondevice 101 and the operation target device 150 also differs.

The camera acquired image sensing unit 110 is controlled by the controlunit 109. The camera acquired image sending unit 110 transmits a cameraacquired image captured by the camera 102 as a part of the detectionresult data 121 to the operation target device 150 via the communicationunit 108.

The operation detectable region specifying unit 107 performs a processsimilar to that performed by the operation detectable region specifyingunit 107 included in the coordinate detection device 101 illustrated inFIG. 1. Namely, the operation detectable region specifying unit 107receives the camera acquired image from the communication unit 151 basedon control from the control unit 153 and specifies the region whereoperation information of a finger is detectable based on the image data.

Also in the operation detection apparatus 100 configured as describedabove, the function and effect similar to those in the aforementionedfirst embodiment can be achieved.

Fifth Embodiment

While the operation detection apparatus 100 illustrated in FIG. 1according to the first embodiment described above is configured to useone camera and two illuminations for detecting the contact pointcoordinates of an electronic pen or the contact point coordinates of afinger, another configuration example will be described in the fifthembodiment.

<Configuration Example of Operation Detection Apparatus>

FIG. 16 is an explanatory view illustrating a configuration example ofan operation detection apparatus 100 according to the fifth embodiment.

The operation detection apparatus 100 illustrated in FIG. 16 isconfigured of the operation target device 150, the camera 102, anelectromagnetic wave generation device 600, and a control device 601. Inthe operation detection apparatus 100 illustrated in FIG. 16, an objectto be detected is a finger in FIG. 16(a), and an object to be detectedis the electronic pen 400 in FIG. 16(b).

The electromagnetic wave generation device 600 is attached to an upperpart of the projection surface 301 or the like and irradiates anelectromagnetic wave in a wavelength region including infrared rays. Theelectromagnetic wave is projected onto a plane so as to be parallel tothe projection surface 301 and along the projection surface 301, and aposition of the object is detected by capturing the electromagnetic wavereflected by an object, which has approached the projection range 302,by the camera 102. The electromagnetic wave to be irradiated may bevisible light such as red light.

The control device 601 controls respective operations of the camera 102,the operation target device 150, and the electromagnetic wave generationdevice 600. Also, components of the operation target device 150, thecamera 102, the electromagnetic wave generation device 600, and thecontrol device 601 may be respectively configured as separate members asillustrated in FIG. 16, or a device in which all or some of thecomponents are integrated may be substituted for the componentsillustrated in FIG. 16.

In the manner described above, it is possible to detect an operationwith the finger of a user on the projection range 302, outputappropriate information from the operation target device 150, andoperate and control the operation target device 150.

In the foregoing, the invention made by the inventors of the presentinvention has been concretely described based on the embodiments.However, it is needless to say that the present invention is not limitedto the foregoing embodiments and various modifications and alterationscan be made within the scope of the present invention.

For example, in the first to fifth embodiments, the user message displayprojected from the video irradiation unit 154 illustrated in FIGS. 8 to11 may be notified to the user by an icon projected from the videoirradiation unit 154.

Alternatively, the user message may be notified to the user bydisplaying a message on the display unit 155. Further, the user messagemay be notified to the user by outputting a voice message or a soundeffect from the audio output unit 157. Even if any of the meansdescribed above is used, a similar effect can be obtained.

Note that the aforementioned embodiments have been described in detailto make the present invention easily understood, and the presentinvention is not limited to that including all the described components.Also, for a part of the configuration of each of the embodiments,addition of another configuration, deletion, and replacement arepossible. Further, some or all of the aforementioned configurations,functions, processing units, processing means, and the like may beimplemented by hardware designed with integrated circuits and the like.

In addition, control lines and information lines considered to berequired in the description are illustrated, and all control lines andinformation lines on a product are not necessarily illustrated. Inpractice, almost all of the components may be considered to be connectedto one another.

REFERENCE SIGNS LIST

-   100 Operation detection apparatus-   101 Coordinate detection device-   102 Camera-   103 Illumination-   104 Illumination-   105 Finger contact position detection unit-   106 Pen contact position detection unit-   107 Operation detectable region specifying unit-   108 Communication unit-   109 Control unit-   110 Camera acquired image sending unit-   120 Detection result data-   121 Detection result data-   150 Operation target device-   151 Communication unit-   152 Setting input unit-   153 Control unit-   154 Video irradiation unit-   155 Display unit-   156 Light emitting unit-   157 Audio output unit-   301 Projection surface-   302 Projection range-   400 Electronic pen-   401 Light emitting element-   402 Tip contact detection unit-   403 Light emission control unit-   600 Electromagnetic wave generation device-   601 Control device-   900 Operation detection determination table

The invention claimed is:
 1. A video display method using a projectionvideo display apparatus being controllable by a user operation of anobject on a projection surface, the video display method comprising:projecting, via a projector, a display video on the projection surface,capturing, via a camera, the projection surface, detecting, via a useroperation detector, the object, via the projection surface, based onimage data captured by the camera specifying, via an operable regionspecifying circuit, each of a first region where the object isdetectable and a second region where the object is undetectable in arange in which the display video is projected on the projection surface,based on luminance levels of the image data captured by the camera;displaying, via the projector, the display video in which the firstregion is distinguished from the second region; and changing, via theprojector, a display position of the object so that the object isdisplayed within a range of the first region where the object via theprojection surface is detectable, and wherein an area of the firstregion and an area of the second region are changed by an influence ofirradiation of external light.
 2. The video display method according toclaim 1, wherein the projector is further configured to: display, viathe projector, an input video input from outside, and change a rangeoccupied by the input video in the display video based on the firstregion specified by the operable region specifying circuit.
 3. The videodisplay method according to claim 1, wherein the operable regionspecifying circuit is further configured to specify a region where theluminance level is higher than a first setting value as the first regionand to specify a region where the luminance level is lower than thefirst setting value as the second region.
 4. The video display methodaccording to claim 3, wherein the method further comprises: grouping,via the operable region specifying circuit, the second region where theobject via the projection surface is undetectable in accordance with theluminance level, and displaying, via the projector, second regiongrouped by the operable region specifying circuit in the display videoso that the second region is identified for each group.
 5. The videodisplay method according to claim 1, wherein the method furthercomprises: detecting, via the operable region specifying circuit, aregion having the luminance level lower than a second setting value asan unusable region where the projection surface does not exist, andsetting, via the projector, a projection range so that the video is notprojected onto the unusable region based on a detection result of theoperable region specifying circuit.
 6. The video display methodaccording to claim 1, wherein the method further comprises: detecting,via the user operation detector, a contact position of at least one of afinger and an electronic pen, and to detect the object via theprojection surface based on the detected contact position.